Combining Nearest Neighbor Classi ers Through Multiple

Combining multiple classiiers is an eeective technique for improving accuracy. There are many general combining algorithms, such as Bagging or Error Correcting Output Coding, that signiicantly improve classiiers like decision trees, rule learners, or neural networks. Unfortunately, many combining methods do not improve the nearest neighbor classiier. In this paper, we present MFS, a combining algorithm designed to improve the accuracy of the nearest neighbor (NN) classiier. MFS combines multiple NN classiiers each using only a random subset of features. The experimental results are encouraging: On 25 datasets from the UCI Repository, MFS sig-niicantly improved upon the NN, k nearest neighbor (kNN), and NN classiiers with forward and backward selection of features. MFS was also robust to corruption by irrelevant features compared to the kNN classiier. Finally, we show that MFS is able to reduce both bias and variance components of error.